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Simple Camel Route 


public class SomeRoute extends RouteBuilder 4 
@Override 
public void configure() 4 
from("direct:start") 
.process( this: :produce); 


“Technology disciplines tend to be objective ... Performance, on 
the other hand, is often subjective ... it can be unclear whether 
there is an issue to begin with, and if so, when it has been fixea” 


Brendan Gregg - Systems Performance 


A computer that is wasting time with 
inefficiencies is still consuming resources. 


Waste leads to more waste. 
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Type Pollution Agent 


e Java Agent 
e Developed by Red Hat AS Performance Team 


e Open Source 


e Identity code that may suffer from type-pollution problems 


Async Profiler 


e Open source low-overhead sampling profiler 
e Works with many JVM distributions 
e Can trace different types of events 

e CPU: cpu 

e Allocations: heap 


e \Wall-clock: time 


Perf 


e Official Linux profiler 
e Has several tools 
e stat: basic statistics about the runtime 
e c2c: cache statistics / false-sharing detection 


e Used as a complement 


JMH 


e Microbenchmark 

e Used interchangeably with other tools 
e lest hypothesis 

e Different benchmark modes 
e Throughput 


e Average time 


e Open source with lots of samples 


Camel Load Generator 


e Macro-benchmark tool 

e long running tests 

e Personal project 

e Designed to stress specific parts of Camel 


e Used along with other tools 


Camel Load Tester -- Type Pollution Agent 


Trail #1 


-javaagent : $AGENT. HOME/type-pollution-agent-0.1-SNAPSHOT. jar=org.apache. camel 


Trail #1 


-javaagent: $AGENT. HOME/type-pollution-agent-0.1-SNAPSHOT. jar=org.apache. camel 


Trail #1 


-javaagent : $AGENT. HOME/type-pollution-agent-0.1-SNAPSHOT. Jarsorg.apache. camel 


Trail #1 


-javaagent : $AGENT. HOME/type-pollution-agent-0.1-SNAPSHOT. jar=org.apache. camel 


Trail #1 


Type-pollution agent 


Last: true 


1: org.apache.camel.support.DefaultExchange 
Count: 4306838067 
Types 


org.apache.camel .ExtendedExchange 
org.apache. camel. Exchange 
Traces: 
org.apache.camel.support.AbstractExchange.adapt(AbstractExchange. java: 607) 
class: org.apache. camel. Exchange 
count: 142594327 
class: org.apache.camel.ExtendedExchange 
count: 42925534 
org.apache.camel.support.ExchangeHelper. copyExchangeAndSetCamelContext(ExchangeHelper. java: 826) 
class: org.apache.camel.ExtendedExchange 
count: 19744262 
org.apache.camel.component.disruptor.DisruptorConsumer..process(DisruptorConsumer. java: 167) 
class: org.apache.camel.ExtendedExchange 
count: 19188788 


Trail #1 


Type-pollution agent 


e 370+ million successful type 
checks for the DefaultExchange 


e 114+ million for the 
DefaultMessage 


e 50+ million for the 
DisruptorProducer 


500 OOOK 


375 000K 


250 000K 


125 000K 


OK 


HH DefaultExchange 
"| DisruptorProducer 


HH DefaultMessage 


Camel Load Tester + Perf stat 


Trail #1 


perf stat -D 5000 java ${LOTS_OF_OPTIONS} -jar your-app. jar 


Trail #1 
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perf stat -D 5000 java $JLOTS OF OPTIONS) -jar your-app. jar 


Trail #1 


۷ 


perf stat -D 5000 java ${LOTS_OF_OPTIONS} -jar your-app. jar 


Trail #1 


perf stat -D 5000 java ${LOTS_OF_OPTIONS} -jar your-app. jar 


\ 


Trail #2 


Perf stat 


Performance counter stats for process id '716275': 


193055.40 msec task-clock H 6.428 CPUs utilized 
1737641 context-switches H 9.001 K/sec 
97 cpu-migrations H 0.502 /sec 
18503 page-faults # 95.843 /sec 
457194305145 cycles H 2.368 GHz 
296078900393 instructions H 0.65 nen per cycle 
54848711045 branches # 284.109 M/sec 
1190686954 branch-misses # 2.17% of all branches 


30.032294196 seconds time elapsed 


Trail #2 


Perf stat 

e Instructions per cycle (IPC) < 1 9,7 
0,6 
0,3 


This is waste. 
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Type check scalability issue 
JDK-8180450 


e Relates to a performance penalty for type checks 
e Has been in the JVM for decades 


e A fix is in progress (hopefully for Java 22) 


Type check scalability issue 
JDK-8180450 


e instanceof 
e Methods: 

e Class::isinstance 

e Class::cast 

e Class::isAssignableFrom 
e checkcast 


e |.e.; generic type erasure 


Given the following code 


Type check scalability issue 
JDK-8180450 


public class SomeType implements T1, T2, 13 4 
j 


Consider the question: “is some 
variable a type Tx” 


Type check scalability issue 
JDK-8180450 


1f (someVar instanceof T2) 4 
System.out.println("This 1s T2"); 
J 


LT (someVar instanceof T1) 4 
System.out.println("This 1s T1"); 
} 


Type check scalability issue 
JDK-8180450 


Secondary Super Cache 


Secondary Supers 


Type check scalability issue 
JDK-8180450 


Secondary Super Cache 
Secondary Supers 


Fields in a class within the JVM 


Type check scalability issue 
JDK-8180450 


Secondary Super Cache 000۷و‎ observed type 


Secondary Supers ڪڪ‎ Array list of known interfaces 


Type check scalability issue 
JDK-8180450 


BD  — 


e 2 11 1 P, 

ıf (someVar ınstanceof T2) 4 Is "someVar T2: 
System.out.println( This is T2"); 

$ 


1f CsomeVar instanceof T1) 4 
System.out.println("This 1s T1"); 
} 


Type check scalability issue 
JDK-8180450 


null 
PEN KK NEDE SE Gê na 
Secondary Super Cache Is “someVar' 12? 


Secondary Supers 


Type check scalability issue 
JDK-8180450 


Is “someVar' T2 


Type check scalability issue 
JDK-8180450 


Is “someVar' 12? 


Type check scalability issue 
JDK-8180450 


e 2 11 1 P, 
1f (someVar instanceof T2) I Is someVar TT: 


System.out.printLnC’ This 1s T2"); 
j 


LT (someVar instanceof T1) 4 
System.out.println("This 1s T1"); 
} 


Type check scalability issue 
JDK-8180450 


T2 


PEN KK NEDE SE Gê na 
Secondary Super Cache Is “someVar' 11? 


Secondary Supers 


Type check scalability issue 
JDK-8180450 


Is “someVar” 117 


Type check scalability issue 
JDK-8180450 


Is “someVar' 117 


Type check scalability issue 
JDK-8180450 


e Ping pong of the cache state 
e May result in an invalidation of the cache line 
e Maintaining cache coherency can be costly 


e Multiple threads can make it worse 


Learn more 
JDK-8180450 


e Talks: 


e Cracking the scalability wall (Java Zone 2023) 
e Cracking the scalability wall (Devoxx UK) 


e Blog posts 


e Seeing through the hardware counters (Netflix blog) 


How does it look like? 


Type check scalability issue 
JDK-8180450 


@Override 


public <T extends CamelContext> T adapt(Class<T> type) 4 
return type.cast(this); 
j 


Type check scalability issue 
JDK-8180450 


Access to restricted APls 


@Override 


public <T extends CamelContext> T adapt(Class<T> type) 4 
return type.cast(this); 
j 


It was pattern ... 


CAMEL-15105 


Create an uniform interface for plugins 


e Create an uniform interface 

e Simplify access to plugins 

e Simplify access to restricted operations 
e CamelContext 


e Exchange 


Defeating the type check scalability monster 
JDK-8180450 


public static PeriodTaskResolver getPeriodTaskResolver(CamelContext camelContext) I 
return getPeriodTaskResolver(camelContext.getCamelContextExtension()); 


} 


Camel Load Tester + Async Profiler 


Trail #3 


-agentpath:/path/to/libasyncProf1ler.so=start,ann,threads,event=cpu,f1le=report.html 


Trail #3 


e 


-agentpath:/path/to/libasyncProf1ler.so=start,ann,threads,event=cpu,f1le=report.html 


Trail #3 


I 


-agentpath:/path/to/libasyncProf1ler.so=start,ann,threads,event=cpu,f1le=report.html 


Trail #3 


-agentpath:/path/to/libasyncProf1ler.so=start,ann,threads,event=cpu,f1le=report.html 


Trail #3 


-agentpath:/path/to/libasyncProf1ler.so=start,ann,threads,event=cpu,f1le=report.html 


Trail #3: producer 


async-profiler 


org.. 

org.. 

org.. 
lo. 8 
org/apache/.. org.. 


 org/apache/c.. org/a.. 


org/apache/camel/impl/.. 
org/apache/camel/impl/.. 
org/apache/camel/proces.. 
e/camel/impl/engine/Cam.. 
mel/component/disrupto.. 
mel/component/disrupto.. 


nel/component/disruptor.. 


otor/BatchEventProcesso.. 
otor/BatchEventProcesso.. 


rent/ThreadPoolExecut.. 
rent/ThreadPoolExecut.. 
d.run 


org.. 

org.. 

org.. 

oo | 
org/apache.. org.. 
 org/apache/.. org/.. 


IMI] or. 
org/apache/camel/impl/engine/C.. 


org/apache/camel/component/disrupt.. 

org/apache/camel/component/disrupt.. 

org/apache/camel/component/disrupt.. 
com/lmax/disruptor/BatchEventProces.. 
com/lmax/disruptor/BatchEventProces.. 


java/util/concurrent/ThreadPoolExec.. 
java/util/concurrent/ThreadPoolExec.. 
java/lang/Thread.run 


org/apache/camel/impl/.. 
or.. org/apache/camel/impl/.. 
org/apache/camel/proce.. 


org.. 
org.. 
org.. 
or.. |o.. 555 | 
۱ or. org/apache/.. org.. 
| or.  org/apache/c.. org/.. 
or.. org/apache/camel/impl/en.. 


or.. org/apache/camel/impl/en.. 
| org.. org/apache/camel/processo.. 
org/apache/camel/impl/engine/Camel.. 


org/apache/camel/component/disruptor/.. 

org/apache/camel/component/disruptor/.. 
org/apache/camel/component/disruptor/D.. 
com/Imax/disruptor/BatchEventProcessor.. 
com/Imax/disruptor/BatchEventProcessor.. 
java/util/concurrent/ThreadPoolExecuto.. 
java/util/concurrent/ThreadPoolExecuto.. 
java/lang/Thread.run 


org.. 
org.. 
| org.. 
| Qe: | oon |i 
| | O.. 'org/apache/.. org/.. 
| | o. _ org/apache/.. org/a.. 


or.. org/apache/camel/impl/e.. 
or. org/apache/camel/impl/e.. 
IN or. 


org/apache/camel/component/disrupto.. 


org/apache/camel/component/disruptor.. 
org/apache/camel/component/disruptor.. 
com/Imax/disruptor/BatchEventProcesso.. 
com/Imax/disruptor/BatchEventProcesso.. 


java/util/concurrent/ThreadPoolExecut.. 
java/util/concurrent/ThreadPoolExecut.. 
java/lang/Thread.run 


org/apache/camel/process.. 
org/apache/camel/impl/engine/Cam.. 


| ۱ 
| 8 | 
۱ ji 
| 
l | 
I | 
la 
3 
com. 
com/. 
com or. | 
H org/ap.. or. or.. 
Il org/ap.. on. or.. 
Il org/ap.. | ES or.. 
Il org/ap.. org/.. or. 
II] org/apache/ca.. or.. 


= org/apache/camel/impl.. 
`. org/apache/camel/impl.. 
= org/apache/camel/impl.. 
` ` org/apache/camel/impl.. 


-= org/apache/camel/support.. 
` org/apache/camel/impl/en.. 
org/apache/camel/impl/eng.. 
org/apache/camel/impl/eng.. 
` org/apache/camel/impl/eng.. 
` org/apache/camel/load/tes.. 
org/apache/camel/load/tes.. 
org/apache/camel/load/tes.. 
` org/apache/camel/load/tes.. 


java/util/concurrent/Exec.. 
java/util/concurrent/Futu.. 
= java/util/concurrent/Thre.. 
` java/util/concurrent/Thre.. 
"| java/lang/Thread.run 


com.. 
com/.. 
com ۰ | or. 
org/ep or. | or. 
org/ap.. or. for. 
om or. or. 
org/ap.. 


org/apache/camel/impl.. 
org/apache/camel/impl.. 
org/apache/camel/impl.. 
org/apache/camel/impl.. 


org/apache/camel/suppor.. 
org/apache/camel/impl/e.. 
org/apache/camel/impl/en.. 
org/apache/camel/impl/en.. 
org/apache/camel/impl/en.. 
org/apache/camel/load/te.. 
org/apache/camel/load/te.. 
org/apache/camel/load/te.. 
org/apache/camel/load/te.. 


java/util/concurrent/Exe.. 
java/util/concurrent/Fut.. 
java/util/concurrent/Thr.. 
java/util/concurrent/Thr.. 
java/lang/Thread.run 


org.. on. 
'org/apache/ca.. or.. 


l 
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com. 

com/. 

come or.. or.. 
org/ap.. or.. or.. 
org/ap.. org.. or.. 
OOo org.. or. 
org/ap.. org/.. or. 


org/apache/ca.. or.. 
org/apache/camel/impl.. 
org/apache/camel/impl.. 
org/apache/camel/impl.. 
org/apache/camel/impl/.. 


org/apache/camel/support.. 
org/apache/camel/impl/en.. 
org/apache/camel/impl/eng.. 
org/apache/camel/impl/eng.. 
org/apache/camel/impl/eng.. 
org/apache/camel/load/tes.. 
org/apache/camel/load/tes.. 
org/apache/camel/load/tes.. 
org/apache/camel/load/tes.. 


java/util/concurrent/Exec.. 
java/util/concurrent/Futu.. 
java/util/concurrent/Thre.. 
java/util/concurrent/Thre.. 
'java/lang/Thread.run 


com.. 

com/. 

com/. 

org/ap.. 

org/ap.. 

org/ap.. 

org/ap.. 
|org/apac 
org/apache/: 
org/apache/ 
org/apache/' 
org/apache/' 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
org/apache/ca 
java/util/concu 
java/util/concu 
java/util/concu 
java/util/concu 
'java/lang/Thre 


The plot thickens 


Trail #3: producer 


async-profiler 


| 

| 

l | 

1 I jdk/interna.. 


d java/util/c.. jdk/inter.. 
۲۲ i java/util/concurren..  java/uti.. java/util.. E S 
NN. fu. java/util/concurren..  java/util.. java/util/con.. | | 9 
sun.. ` ۰ java/util/concurrent/locks/Ree.. java/util/concur.. org/apache/cam.. org/apache/ca.. 
com..  java/uti.. java/util/concurrent/locks/Ree.. java/util/concurr.. org/apache/came.. org/apache/ca.. 
com/Imax/disruptor/Mu.. com/Imax/disruptor/BlockingWaitStrategy.signalAllWhenBI.. org/apache/camel/s.. org/apache/cam.. 
com/lmax/disruptor/Mu.. com/lmax/disruptor/MultiProducerSequencer.publish org/apache/.. | org/apache/camel/supp.. 'org/apache/camel/.. org/apache/camel/.. 
com/Imax/disruptor/Ri.. com/Imax/disruptor/RingBuffer.publish or.. org/apache/camel/impl/engine/DefaultUnitOfWo.. i org/apache/camel/impl/engine/DefaultUnitOfW.. 
org/apache/camel/component/disruptor/DisruptorReference.publishExchangeOnRingBuffer or.. org/apache/camel/support/UnitOfWorkHelper.done.. ` 0..  org/apache/camel/impl/engine/DefaultUnitOfW.. 
org/apache/camel/component/disruptor/DisruptorReference.publish org..  org/apache/camel/impl/engine/CamellnternalProce.. org/apa.. org/apache/camel/impl/engine/DefaultUnitOfW.. 
org/apache/camel/component/disruptor/DisruptorEndpoint.publish org.. org/apache/camel/impl/engine/CamelInternalProce.. org/apac.. org/apache/camel/impl/engine/CamellnternalPr.. 
org/apache/camel/impl/engine/CamelInternalPro.. 


org/apache/camel/impl/engine/SharedCamelInternalProcessor$.. 


org/apache/camel/component/disruptor/DisruptorProducer.doPublish org.. 
_ org/apache/camel/component/disruptor/DisruptorProducer.process 
org/apache/camel/impl/engine/SharedCamelInternalProcessor.process 
org/apache/camel/impl/engine/SharedCamellnternalProcessor$1.process 
org/apache/camel/impl/engine/DefaultAsyncProcessorAwaitManager.process or.. 
org/apache/camel/impl/engine/SharedCamelInternalProcessor.process org/.. 
org/apache/camel/support/cache/DefaultProducerCache.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.sendBody 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate.produceMessages 
org/apache/camel/load/tester/routes/DisruptorVMThreadedProducerTemplate.produceMessages 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate.lambda$produce$0 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate$$Lambda$264.0x00007fedc8241a08.run 
java/util/concurrent/Executors$RunnableAdapter.call 
java/util/concurrent/FutureTask.run 
java/util/concurrent/ThreadPoolExecutor.runWorker 
java/util/concurrent/ThreadPoolExecutor$Worker.run 
java/lang/Thread.run 
[pool-2-thread-1 tid=718184] 


all 
Matched: 12.23% X 


org/apache/camel/impl/engine/CamelInternalPro.. 


Trail #3: producer 


async-profiler 


| 

| | au 

LJ 5۱ & 

1 I jdk/interna.. 
i d java/util/c.. jdk/inter.. 
۲۲ i java/util/concurren.. java/uti.. java/util.. 
MM fu. java/util/concurren..  java/util.. java/util/con.. 
sun.. jdk/inte.. java/util/concurrent/locks/Ree.. java/util/concur.. 
com..  java/uti.. java/util/concurrent/locks/Ree.. java/util/concurr.. 
com/Imax/disruptor/Mu.. com/Imax/disruptor/BlockingWaitStrategy.signalAllWhenBI.. 


com/Imax/disruptor/Mu.. com/Imax/disruptor/MultiProducerSequencer.publish 
com/Imax/disruptor/Ri.. com/Imax/disruptor/RingBuffer.publish 
org/apache/camel/component/disruptor/DisruptorReference.publishExchangeOnRingBuffer 
org/apache/camel/component/disruptor/DisruptorReference. publish 
org/apache/camel/component/disruptor/DisruptorEndpoint.publish 
org/apache/camel/component/disruptor/DisruptorProducer.doPublish 

_ org/apache/camel/component/disruptor/DisruptorProducer.process 


org/apache/camel/impl/engine/SharedCamelInternalProcessor.process 
org/apache/camel/impl/engine/SharedCamelInternalProcessor$1.process 
org/apache/camel/impl/engine/DefaultAsyncProcessorAwaitManager.process 
org/apache/camel/impl/engine/SharedCamelInternalProcessor.process 
org/apache/camel/support/cache/DefaultProducerCache.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.send 
org/apache/camel/impl/engine/DefaultProducerTemplate.sendBody 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate.produceMessages 
org/apache/camel/load/tester/routes/DisruptorvMThreadedProducerTemplate.produceMessages 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate.lambda$produce$0 
org/apache/camel/load/tester/routes/ThreadedProducerTemplate$$Lambda$264.0x00007fedc8241a08.run 
java/util/concurrent/Executors$RunnableAdapter.call 
java/util/concurrent/FutureTask.run 
java/util/concurrent/ThreadPoolExecutor.runWorker 
java/util/concurrent/ThreadPoolExecutor$Worker.run 
java/lang/Thread.run 
[pool-2-thread-1 tid=718184] 
all 


8 ۳ 
5 ۳ 
org/apache/cam.. org/apache/ca.. 
org/apache/came.. org/apache/ca.. 
org/apache/camel/s.. org/apache/cam.. 
'org/apache/.. | org/apache/camel/supp.. 'org/apache/camel/.. org/apache/camel/.. 
or.. org/apache/camel/impl/engine/DefaultUnitOfWo.. i org/apache/camel/impl/engine/DefaultUnitOfW.. 
or.. org/apache/camel/support/UnitOfWorkHelper.done.. ` 0..  org/apache/camel/impl/engine/DefaultUnitOfW.. 
org..  org/apache/camel/impl/engine/CamellnternalProce.. org/apa.. org/apache/camel/impl/engine/DefaultUnitOfW.. 
org.. org/apache/camel/impl/engine/CamelInternalProce.. org/apac.. org/apache/camel/impl/engine/CamelInternalPr.. 
org.. org/apache/camel/impl/engine/SharedCamelInternalProcessor$.. org/apache/camel/impl/engine/CamelInternalPro.. 


org/apache/camel/impl/engine/CamelInternalPro.. 


or.. 
org/.. 


Matched: 12.23% X 


Trail #3: consumer 


async-profiler 


| org/apa.. | org/apache.. 


org/apache/camel/impl/en.. 


or.. 
or.. 


i 5 
liz 
org/.. 
org/.. 
org/.. 
org/.. 


org/apach.. 
d i org/apach.. 
j org/apache/.. 
org/apac.. 'org/ap.. org/apache/c.. 
org/apac.. org/apache/camel/impl/engine/Defa.. 
org/apach.. ja.. OG org/apache/camel/support/UnitOfWor.. 
ors org/apache/camel/impl/engine/Camell.. 
HUE org/apache/camel/.. org.. org/apache/camel/impl/engine/Camell.. 

` org/apache/camel/impl/engine/CamelinternalProcessor$AsyncAfterTask.done 


i org/apache/camel/impl/eng.. 

Å org/apache/camel/impl/eng.. “java.. j.. org/apache/camel/AsyncCallback.run 
o.. org/a.. Be org/apache/camel/impl/eng.. org/apache/camel/impl/engine/DefaultReactiveExecutor$Worker.schedule 
o.. org/a.. 'org/apac.. o.. org/apache/camel/impl/engine.. org/apache/camel/impl/engine/DefaultReactiveExecutor.scheduleMain 
org/apache/c..  org/apache/ca.. o. org/apache/camel/impl/engine/..  org/apache/camel/processor/Pipeline.process 
org/apache/c.. org/apache/camel/impl/engine/CamelInternalProcessor.process 


ache/camel/component/disruptor/DisruptorConsumer.process 


iche/camel/component/disruptor/DisruptorConsumer$ConsumerEventHandler.onEvent 
/camel/component/disruptor/DisruptorConsumer$ConsumerEventHandler.onEvent 


tor/BatchEventProcessor.processEvents 
tor/BatchEventProcessor.run 
ent/ThreadPoolExecutor.runWorker 
ent/ThreadPoolExecutor$Worker.run 
run 


E O.. 
org/apache/cam.. 
org/apache/cam.. 
org/apache/cam.. les 
org/apache/came.. org.. 

org/.. org/apache/camel/i.. org.. 
org/ap.. | org/apache/camel/im.. or.. org/.. 


org/apache/camel/impl/engine/Camellntern.. 


org/apache/camel/processor/Pipeline$PipelineTas.. 


Mai 


Trail #3: consumer 


async-profiler 


org/apache/camel/impl/en.. 


or.. 

or.. 

1 ku 

BES 

org/apach.. org/.. 

d l org/apach.. org/.. 

j org/apache/.. org/.. 
org/apac.. org/ap.. | org/apache/c.. org/.. 
org/apac.. org/apache/camel/impl/engine/Defa.. 
org/apach.. ja.. OG org/apache/camel/support/UnitOfWor.. 
org/apache.. or.. org/apache/camel/impl/engine/Camell.. 
HUE org/apache/camel/.. org.. org/apache/camel/impl/engine/Camell.. 


` org/apache/camel/impl/engine/CamelinternalProcessor$AsyncAfterTask.done 


Û org/apache/camel/impl/eng.. 
Å org/apache/camel/impl/eng.. “java.. j.. org/apache/camel/AsyncCallback.run 
o.. org/di. Be org/apache/camel/impl/eng.. org/apache/camel/impl/engine/DefaultReactiveExecutor$Worker.schedule 


org/apac.. o.. 


org/apache/camel/impl/engine.. 


org/apache/camel/impl/engine/DefaultReactiveExecutor.scheduleMain 
org/apache/camel/processor/Pipeline.process 


o.. org 
org/apache/c..  org/apache/ca.. o.. org/apache/camel/impl/engine/.. 
org/apache/c.. org/apache/camel/impl/engine/CamelInternalProcessor.process 


ache/camel/component/disruptor/DisruptorConsumer.process 


iche/camel/component/disruptor/DisruptorConsumer$ConsumerEventHandler.onEvent 
/camel/component/disruptor/DisruptorConsumer$ConsumerEventHandler.onEvent 


tor/BatchEventProcessor.processEvents 
tor/BatchEventProcessor.run 
ent/ThreadPoolExecutor.runWorker 
ent/ThreadPoolExecutor$Worker.run 
run 


E O.. 
org/apache/cam.. 
org/apache/cam.. 
org/apache/cam.. les 
org/apache/came.. org.. 

org/.. org/apache/camel/i.. org.. 
org/ap.. | org/apache/camel/im.. or.. org/.. 


org/apache/camel/impl/engine/Camellntern.. 


org/apache/camel/processor/Pipeline$PipelineTas.. 


Mai 


Trail #3 


async-profiler 


e Methods doing unnecessary work 
e Ĉontention 
e LongAdder instead of Atomiclnteger 
e More type-pollution problems 
e Large methods hiding valuable information 


e Likely also preventing inlining 


Monster slaying strategy 


Too many monsters and they move fast 


e Automate 
e Fast moving target 
e Watch constantly 


e View the trends 


Monster slaying challenges 
Benchmarking is difficult 


e Variations across the stack 
e Impacts from system architecture (i.e.: NUMA, core-to-core latency, etc) 
e Type pollution issue: 


e (Approximately) 1 in 8 chance to happen 


Collecting the rewards 


e Zero type checks for 500 OOOK 
e Critical classes 375 000K 

e In the hot path” 250 000K 
125 000K 

OK 


HH DefaultExchange 
"| DisruptorProducer 


HH DefaultMessage 


Collecting the rewards 


Improved system usage 


e Camel 4.0.1 1,2 


e |mproved IPC 08 


e Camel 4.1.0-SNAPSHOT: 
0,4 
e Small regression 


e Under investigation 9 


Collecting rewards 
JMH 


e Camel 4.0.1 
e Ĉonsistently better than 3.x 
e Camel 4.1.0 
e On par with 4.0.x with a few regressions 


e Need to reduce the uncertainty in a few scenarios 


Collecting rewards 


Camel Load Tester 


e Throughput is better 
e Lomponents 
e Disruptor 


e Seda 


© Slower 


Disruptor: 4.0.0 x 3.20.4 


Collecting rewards 


Camel Load Tester 


e Patterns (4.1.0) 
e Content-based-router 
e Filter 
e Aggregator 


e (Ore 


© Faster © Slower 


CBR: 4.1.0-SNAPSHOT x 4.0.0 


Collecting rewards 


Camel Load Generator 


e Components e HW 
e Disruptor e 2x Intel Xeon Silver 4116 CPU E 
2.10GHz 
e Seda 
e 12c/24t 
e Camel 3.20.7-SNAPSHOT 
e Java 
e Right after vote 
e 17.0.8 


e -XX:+UseNUMA -Xmx4G 


Collecting rewards 


e Some tools can uncover hidden bottlenecks 
e Many times the fixes are simple 

e If they are not part of a pattern 
e Benchmarking is hard 


e JMH simplifies a lot 


Find me online 
Otavio R. Piske 


